Structured Representations in Visual Working Memory

How much visual information can we hold in mind at once? A large body of research has attempted to quantify the capacity of visual working memory by focusing on how many individual objects or visual features can be actively maintained in memory. This thesis presents a novel theoretical framework for understanding working memory capacity, suggesting that our memory representations are complex and structured even for simple visual displays, and formalizing such structured representations is necessary to understand the architecture and capacity of visual working memory. Chapter 1 reviews previous empirical research on visual working memory capacity, and argues that an understanding of memory capacity requires moving beyond quantifying how many items people can remember and instead focusing on the content of our memory representations. Chapter 2 argues for structured memory representations by demonstrating that we encode a summary of all of the items on a display in addition to information about particular items, and use both item and summary information to complete working memory tasks. Chapter 3 describes a computational model that formalizes the roles of perceptual organization and the encoding of summary statistics in visual working memory, and provides a way to quantify capacity even in the presence of richer, more structured memory representations. This formal framework predicts how well observers will be able to remember individual working memory displays, rather than focusing on average performance across many displays. Chapter 4 uses information theory to examine visual working memory through the framework of compression, and demonstrates that introducing regularities between items allows us to encode more colors in visual working memory. Thus, working memory capacity needs to be understood by taking into account learned knowledge, rather than simply focusing on the number of items to be remembered. Together, this research suggests that visual working memory capacity is best characterized by structured representations where prior knowledge influences how much can be stored and displays are encoded at multiple levels of abstraction. Thesis Supervisor: Aude Oliva Title: Associate Professor of Cognitive Science